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Abstract 



Since Malone advanced his theory of intrinsically motivating instruction against the backdrop of 
digital gaming (1981), some evidence has been put forth that serious games may increase 
intrinsic motivation for iearning (de Freitas & Griffiths, 2007; Dickey, 2007). In this 
presentation, we wiil report on the LLINGO project, which aims to develop a prototype for 
language learning in immersive, game-based environments, by pitting instructional design 
against game design. From an instructional design perspective, we will tackle various aspects 
such as the need for a task-based approach, learner control and the role of learner 
perceptions. From the point of view of game design, we will discuss story and scripting, 
character development and game mechanics. Finally, we will discuss this design framework in 
light of the targeted users. 
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Introduction 

One of the fundamental aspects of language testing is its effect in teaching. Currently, 
the Spanish Government is researching the possible implications of implementing an Internet 
based language test for the national University Entrance Examination (PAULEX project 
HUM2007-66479-C02-01/FILO). There are a number of issues that have concerned the project 
researchers in relation to what kind of tasks would be considered to show more significant 
differences from pen-and-paper and computer based tests. Soon in the research, the team 
found that their main concern should be the writing and speaking tasks. Although extensive 
study in relation to oral tasks has been done by testing companies such as the Educational 
Testing Service (TOEFL) and NCS Person e-Measurement Services (Pearson PTE), for the 
Spanish researchers the main concern was posed on the written tasks. This was so for two 
main reasons: (1) the oral tasks have not implemented in the University Entrance Examination 
yet (scheduled for 2012), and (2) extensive trialing is necessary and give the current funds, it 
may take two years to test a minimal sample to take decisions. In relation to the writing 
processes, the research team first addressed issues such as web design (Garcia Laborda, 
2009); ergonomics (Garcia Laborda et al, 2010); scoring and benchmarking processes or 
experimental design. After doing all this previous work, in November 2009, 260 students took 
a computer based version of the University Entrance Examination in Valencia (Spain). 

After completion of the test, the research team proceeded to two main assessments: a holistic 
one and an error analysis. Both analyses indicated a number of issues: 

1) The holistic assessment indicated that the results of both the group and the individuals 
was not very different from the results obtained by the candidates who took the same 
test only a few before; 

2) Students were happy about the testing platform and felt that it was a reliable way to 
measure their language proficiency; 
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3) Errors due to computer use and typos were a small percentage of the total number of 

errors and the average word count was also similar to the control test (the one that was 
used) 

Computer processes: Observation results 

Observation has hardly ever considered as a reliable method of research (at the most a quasl- 
experimental method). However, In this case. It was considered in order to observe the 
students' reactions to the test. Two 1 hour sessions were recorded. However, due to the 
Spanish rights of privacy, little material can be used actually. The writing speed and their body 
language expressions were also annotated (Garcia Laborda & Gimeno Sanz, 2010). According 
to the observations, the team obtained three main conclusions: 

1) Students showed no sign of additional difficulty. In general, with a few exceptions, 
students did not require much support to work. Most assistance, in fact, was provided 
to support the students with the software deficiencies (mostly due to connectivity); 

2) Students worked regularly and typed at an adequate speed to fulfill the main written 
task in the time given; 

3) Their keyboard use was adequate, at least, for the tasks they were assigned. 

Educational changes associated to the trialing stage 

By this experimentation, the research team obtained evidence to support the implementation 
of the Computer based University Entrance examination with a few premises: 

1) It is necessary to motivate both teachers and students to accept, understand and 
integrate the change. Garcia Laborda (2010) verified that although teachers may be 
reluctant to change at the beginning, they may also be willing to take the necessary 
training that may help them to understand and accept their attitudes' change. 

2) Effective and simple applications are necessary to motivate students. The team believes 
that simplicity in interface and flow design may have a significant effect in students 
accepting the application. If writing was considered to be the most troublesome aspect 
in the change, as it was observed, when students feel at ease they may be prone to 
perform, at least, as well In a computer environment as they would in a different 
context (pen and paper). According to Weir (2005), delivery context may be a crucial 
factor In determining the validity of a language test. 

3) Motivation should also be an Important aspect to consider for the change. Since foreign 
language teachers in Spain tend to be reluctant to integrate computers in writing 
(although they may be more willing to use it with multiple choice exercises or reading 
activities), it is important to facilitate meaningful training. 

Overall, the results obtained in the experimental stage of the project may motivate dramatic 
educational changes. Therefore, as observed the teachers' and students' reconceptualization 
may be supported by three main pillars: knowledge of the software, acquisition of working 
strategies (both for teaching and learning), and security that the results will be reliable and, at 
least, as good in the new delivery context. 

Even more important could be these results in light of their institutional effects and the effects 
in the regular teaching (washback). Since students can adapt easily to the new delivery 
context, it could be plausible believing that with some economical effort computer testing can 
benefit economically the educational boards around Spain on the medium term. 

Conclusions 

If students are able to adapt themselves to the written section of computer assisted language 
tests without previous training, there is little doubt that they could adapt to the rest of the 
test. There is also little debate on whether computers can be used to improve the assessment 
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process while reducing the costs related to testing. This facility should motivate the 
incorporation of computer based testing platforms for the University Entrance examination. 

The researchers would like to express their gratitude to the Ministry of Education of Spain for 
supporting the development and implementation of the testing platform hereby mentioned 
under the PAER project (HUM2007-66479-C02-01/FILO). 
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